Kubernetes Topology Spread Constraints

What are Topology Spread Constraints?

Topology Spread Constraints help Kubernetes distribute application pods across different topology domains, such as nodes or availability zones. This improves workload resilience by reducing the chance that too many replicas are affected by a single infrastructure failure.

Real-world example: 3 nodes and 6 replicas

Imagine an AKS cluster with three eligible nodes running six replicas of a critical microservice.

Without an appropriate spread policy

Node 1: 4 pods
Node 2: 1 pod
Node 3: 1 pod

Pods may be distributed unevenly. If Node 1 fails, several replicas could become unavailable at the same time.

With Topology Spread Constraints

Node 1: 2 pods
Node 2: 2 pods
Node 3: 2 pods

With all three nodes eligible and sufficient capacity, Kubernetes aims to keep the pod counts evenly spread. If one node fails, replicas on the other nodes can continue serving traffic while Kubernetes works to recover the affected replicas.

Example Kubernetes Deployment configuration

Add the following configuration under spec.template.spec in your Deployment. The pod labels must match the labelSelector.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: critical-app
spec:
  replicas: 6
  selector:
    matchLabels:
      app: critical-app
  template:
    metadata:
      labels:
        app: critical-app
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: critical-app
      containers:
        - name: app
          image: nginx:stable

kubernetes.io/hostname spreads matching pods across nodes. For zone-level spreading, use topology.kubernetes.io/zone, provided the cluster nodes have the appropriate zone labels.

How can we minimize application impact when a node fails?

1. Topology Spread Constraints

Spread replicas across different nodes or availability zones so a single failure is less likely to affect every replica.

2. Run multiple replicas

Use a Deployment with multiple replicas. A single pod cannot provide the same level of availability as a properly distributed set of replicas.

3. Configure readiness probes

Readiness probes help Kubernetes determine whether a pod is ready to receive traffic. Services normally send traffic only to ready endpoints.

4. Use a Pod Disruption Budget (PDB)

A PDB helps limit disruptions during supported voluntary operations, such as node maintenance. It does not prevent involuntary failures, such as a node crashing.

5. Spread critical workloads across availability zones

For important applications, consider zone-level spreading as well as node-level spreading. This can reduce the impact of a full availability-zone outage.

6. Maintain sufficient cluster capacity

Ensure the remaining nodes have enough CPU and memory to run replacement pods. Cluster autoscaling may help add capacity, but provisioning and scheduling take time.

Key parameters to remember

• maxSkew: 1 — limits the permitted difference in matching pod counts across eligible topology domains for this constraint.
• topologyKey — identifies the topology domain, such as a node hostname or availability zone.
• DoNotSchedule — leaves a pod unscheduled if placing it would violate the constraint.
• ScheduleAnyway — prefers placements that improve the spread, but allows scheduling even when the ideal spread cannot be achieved.